Image manipulation with natural language using Two-sided Attentive Conditional Generative Adversarial Network

Abstract Altering the content of an image with photo editing tools is a tedious task for inexperienced user, especially, when modifying visual attributes specific object in without affecting other constituents such as background etc. To simplify process manipulation and to provide more control users, it better utilize simpler interface like natural language. It also enables semantically modify parts according given text. Therefore, this paper, we address challenge manipulating images using language descriptions. We propose Two-sidEd Attentive conditional Generative Adversarial Network (TEA-cGAN) generate manipulated images. TEA-cGAN’s contribution seen two-fold. The first aims attend locations that need be modified during generation. introduces two types architectures fine-grained attention both generator discriminator (GAN). specific, one i.e., Single-scale architecture used focuses only text-relevant regions leaves untouched. While second Multi-scale further extended idea by taking different scales features into account. purpose higher resolution (e.g., 256 × 256) they quality stability. Quantitative qualitative experiments conducted on CUB Oxford-102 datasets confirm TEA-cGAN scale outperform existing methods while generating 128 × 128 including 256 × 256.